System and method for navigating a tomosynthesis stack using synthesized image data

ABSTRACT

A system and method for displaying and navigating breast tissue is configured for or includes obtaining a plurality of 2D and/or 3D images of a patient&#39;s breast; generating a synthesized 2D image of the breast from the obtained images; displaying the synthesized 2D image; receiving a user command, or otherwise detecting through a user interface, a user selection or other indication of an object or region in the synthesized 2D image; and displaying at least a portion of one or more images from the plurality, including a source image and/or most similar representation of the user selected or indicated object or region.

RELATED APPLICATION DATA

This application is a continuation of U.S. patent application Ser. No.16/555,925, filed on Aug. 29, 2019, which is a continuation of U.S.patent application Ser. No. 15/794,635, filed on Oct. 26, 2017, nowissued as U.S. Pat. No. 10,410,417, which is a continuation of U.S.patent application Ser. No. 14/376,530, now issued as U.S. Pat. No.9,805,507, which is a National Phase Entry under 35 U.S.C. § 371 ofInternational Patent Application No. PCT/US2013/025993, having aninternational filing date of Feb. 13, 2013, which claims the benefit ofpriority under 35 U.S.C. § 119 to U.S. Provisional Patent ApplicationNo. 61/597,958, filed Feb. 13, 2012, and which is hereby incorporated byreference in its entirety.

FIELD

The inventions disclosed herein pertain to breast imaging usingtomosynthesis, and more specifically to a system and method for guidingthe navigation of a tomosynthesis data set, which employs a synthesized2D image that is obtained by importing relevant data from thetomosynthesis data set into the synthesized image, and then using the 2Dimage to navigate the tomosynthesis data.

BACKGROUND

Mammography has long been used to screen for breast cancer and otherabnormalities. Traditionally, mammograms have been formed on x-ray film.More recently, flat panel digital imagers have been introduced thatacquire a mammogram in digital form, and thereby facilitate analysis andstorage of the acquired images, and provide other benefits as well.Further, substantial attention and technological development has beendedicated towards obtaining three-dimensional images of the breast,using methods such as breast tomosynthesis. In contrast to the 217images generated by legacy mammography systems, breast tomosynthesissystems construct a 3D image volume from a series of 2D projectionimages, each projection image obtained at a different angulardisplacement of an x-ray source relative to the image detector as thex-ray source is scanned over the detector. The constructed 3D imagevolume is typically presented as a plurality of slices of image data,the slices being geometrically reconstructed on planes parallel to theimaging detector. The reconstructed tomosynthesis slices reduce oreliminate the problems caused by tissue overlap and structure noisepresent in single slice, two-dimensional mammography imaging, bypermitting a medical professional (e.g., a radiologist) to scrollthrough the image slices to view underlying structures.

Tomosynthesis systems have recently been developed for breast cancerscreening and diagnosis. In particular, Hologic, Inc. (www.hologic.com),has developed a fused, multimode mammography/tomosynthesis system thatacquires one or both types of mammogram and tomosynthesis images, eitherwhile the breast remains immobilized or in different compressions of thebreast. Other companies have proposed the introduction of systems whichare dedicated to tomosynthesis imaging; i.e., which do not include theability to also acquire a mammogram.

However, systems restricted to tomosynthesis acquisition and imagedisplay may present an obstacle to acceptance of the tomosynthesisimaging technology, as medical professionals have grown accustomed toscreening and analysis of conventional 2D mammogram images. Inparticular, mammograms provide good visualization ofmicro-calcifications, and can offer higher spatial resolution whencompared with tomosynthesis images. While tomosynthesis images providedby dedicated breast tomosynthesis systems have other desirablecharacteristics, e.g., better isolation and visualization of structuresin the breast, such systems do not leverage the existing interpretationexpertise of medical professionals.

Examples of systems and methods that leverage existing medical expertisein order to facilitate the transition to tomosynthesis technology aredescribed in U.S. Pat. No. 7,760,924, which is hereby incorporated byreference in its entirety. In particular, U.S. Pat. No. 7,760,924describes a method of generating a synthesized 2D image, which may bedisplayed along with tomosynthesis projection or reconstructed images,in order to assist in screening and diagnosis.

SUMMARY

According to one aspect of the inventions disclosed and describedherein, a system and method for processing, displaying and navigatingbreast tissue information is provided, wherein the system is configuredfor, and the method includes: (i) obtaining a plurality of 2D and/or 3Dimages of a patient's breast; (ii) generating a synthesized 2D image ofthe patient's breast from the obtained 2D and/or 3D images of theplurality; (iii) displaying the synthesized 2D image; (iv) receiving auser command, or otherwise detecting through a user interface, a userselection or other indication of an object or region in the synthesized2D image; and (v) displaying at least a portion of one or more imagesfrom the plurality, including a source image and/or most similarrepresentation of the user selected or indicated object or region.

Additionally and/or alternatively, the system may be configured for, andthe method may include, concurrently displaying a respective sourceimage and/or most similar representation of a tissue structure or regionthat corresponds to a given location of a user movable input device inthe displayed synthesized 2D image. While various image processingtechniques may be employed for providing the this navigationalfunctionality, in a preferred embodiment, the system is preferablyconfigured for, and the method further includes, generating an index mapcomprising identifying information of selected images of the pluralityof 2D and/or 3D images that are source images or that otherwise containa most similar representation of regions and/or objects displayed in thesynthesized 2D image. The index map can thereafter be used by the systemfor to greatly reduce the time needed to navigate through the images,e.g., a tomosynthesis volume stack of the breast image volume.

The plurality of source images may include one or more of tomosynthesisprojection images, reconstructed tomosynthesis slices, mammograms,contrast enhanced mammograms, and synthesized two dimensional images. Invarious embodiments, the plurality of 2D and/or 3D images of a patient'sbreast are acquired or synthesized X,Y coordinate slices at differing zaxis locations of the breast, the images having one or morecorresponding X,Y coordinate locations. In one embodiment, generatingthe synthesized 2D image includes constructing a merged image byimporting one or more objects and/or regions from the images of theplurality into the merged image, wherein an image from which an objector region is imported into the merged image comprises a source image forthat object or region. In such embodiment, objects or regions arepreferably imported into the merged image at X,Y coordinate locationscorresponding to the X,Y coordinate locations of the respective objectsor regions in their source image. Further to such embodiment, each imageof the plurality of 2D and/or 3D images preferably contains one or moreregions defined by their X,Y coordinate locations that are common forall images of the plurality, wherein one of each said common region isimported from the plurality of images into the merged image based upon acomparison of one or more system and/or user defined attributes of therespective common region of each image.

In a preferred variation of this embodiment, an identified object orregion of interest in a given image has priority for importation intothe merged image over any other identified objects or regions ofinterest having the same or overlapping X,Y coordinate locations inother image slices based upon a predefined priority scheme, e.g., toreflect the relative clinical importance of the various possible tissuestructures. The preferred attributes may include attributes indicativeof regions of interest, such as cancers, or alternatively such as moreaccurate representation of breast density or breast anatomy, i.e.,truthful breast-border/nipple appearance, or presence of a contrastagent in the case of contrast enhanced mammography. In general, anyattribute capable to delivering a high/better-quality image can berelevant.

In various embodiments, an object or region may be automaticallyhighlighted in the synthesized 2D image and/or displayed at leastportion of the one or more images from the plurality. Additionallyand/or alternatively, an object or region in the synthesized 2D imageand/or displayed at least portion of the one or more images from theplurality may be highlighted in response to a further received usercommand or to certain user activity detected through the user interface.By way of non-limiting example, an object or region may is highlightedby a contour line representing a boundary of the highlighted object orregion. Preferably, the object or region is highlighted in a mannerindicating that the highlighted object or region is or contains aspecified type of tissue structure.

According to another aspect of the inventions disclosed and describedherein, a system and method for processing, displaying and navigatingbreast tissue information is provided, wherein the system is configuredfor, and the method includes: (i) obtaining a plurality of tomosynthesisimages comprising volumetric image data of a patient's breast; (ii)generating a synthesized 2D image of the patient's breast at least inpart from the tomosynthesis images; (iii) displaying the synthesized 2Dimage; (iv) receiving a user command, or otherwise detecting through auser interface, a user selection or other indication of an object orregion in the synthesized 2D image; and (v) displaying at least aportion of one or more tomosynthesis images from the plurality,including a source image and/or most similar representation of the userselected or indicated object or region. Again, while various imageprocessing techniques may be employed for providing the thisnavigational functionality, in a preferred embodiment, the system ispreferably configured for, and the method further includes, generatingan index map that includes identifying information of selectedtomosynthesis images of the plurality that are source images or thatotherwise contain a most similar representation of regions and/orobjects in the synthesized 2D image.

These and other aspects and embodiments of the disclosed inventions aredescribed in more detail below, in conjunction with the accompanyingfigures.

BRIEF DESCRIPTION OF FIGURES

FIG. 1 is a block diagram illustrating the flow of data through a systemthat includes a combination mammography/tomosynthesis acquisition systemand/or a tomosynthesis-only acquisition system to acquire tomosynthesisand/or mammography (including contrast mammography) images of apatient's breast, and further includes one or more processors thatimplement the image merge technology of the presently disclosedinventions for providing a two dimensional synthesized image byimporting the most relevant data from the acquired 2D and/or 3D sourceimages into a single merged 2D image for display to a medicalprofessional;

FIG. 2 is a diagram illustrating the data flow of a series oftomosynthesis slices and a synthesized 2D mammogram through the imagemerge technology of the presently disclosed inventions to generate amerged image and a corresponding merge (or “guidance”) map;

FIG. 3 depicts one embodiment of a displayed merged image, whereincertain region boundaries are dynamically identified during merge imagebuild;

FIG. 4 is a flow diagram illustrating exemplary steps performed duringan image merge process according to one embodiment of the presentlydisclosed inventions;

FIGS. 5A and 5B illustrate one embodiment of a display of a mergedimage, and a resultant display of a source image in response toselection of a region in the merged image by a user;

FIG. 6 is a flow diagram illustrating an exemplary process forretrieving and presenting a reconstructed tomosynthesis image slice inresponse to user-selection of an object of interest in a synthesized 2Dimage, according to one embodiment of the presently disclosedinventions;

FIG. 7 is a flow diagram illustrating another exemplary process forretrieving and presenting a reconstructed tomosynthesis image slice inresponse to user-selection of an object of interest in a synthesized 2Dimage, according to another embodiment of the presently disclosedinventions;

FIG. 8 is a flow diagram illustrating a process for constructing acomposite index map of a synthesized 2D image to a correspondingreconstructed tomosynthesis image stack, according to still anotherembodiment of the presently disclosed inventions;

FIG. 9 depicts an exemplary user interface, including a left-hand sidemonitor displaying a synthesized 2D image of a patient's breast,including a highlighted tissue structure, wherein the highlighting is inthe form of a contour line that represents a boundary of the highlightedtissue structure, and a right-hand side monitor displaying thetomosynthesis image from which the highlighted tissue structure wasimported into the 2D image, or which otherwise provides a best view ofthe highlighted tissue structure;

FIG. 10 depicts the user interface of FIG. 9, again displaying asynthesized 2D image of a patient's breast including a highlightedspiculated mass in the left-hand monitor, and a right-hand side monitordisplaying the tomosynthesis image from which the depicted spiculatedmass was imported into the 2D image, or which otherwise provides a bestview of the spiculated mass; and

FIG. 11 depicts the user interface of FIG. 10, including the same breastimage displayed in the left-hand side monitor, but now highlighting aregion containing micro-calcifications, with the right-hand side monitordisplaying the tomosynthesis image from which the highlighted regioncontaining the micro-calcifications was imported into the 2D image, orwhich otherwise provides a best view of the micro-calcifications.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

In describing the depicted embodiments of the disclosed inventionsillustrated in the accompanying figures, specific terminology isemployed for the sake of clarity and ease of description. However, thedisclosure of this patent specification is not intended to be limited tothe specific terminology so selected, and it is to be understood thateach specific element includes all technical equivalents that operate ina similar manner. It is to be further understood that the variouselements and/or features of different illustrative embodiments may becombined with each other and/or substituted for each other whereverpossible within the scope of this disclosure and the appended claims.

The following abbreviations shall have the following definitionsthroughout this patent specification:

Mp refers to a conventional mammogram or contrast enhanced mammogram,which are two-dimensional (2D) projection images of a breast, andencompasses both a digital image as acquired by a flat panel detector oranother imaging device, and the image after conventional processing toprepare it for display and/or storage or other use.

Tp refers to an image that is similarly two-dimensional (2D), but isacquired at a respective tomosynthesis angle between the breast and theorigin of the imaging x rays (typically the focal spot of an x-raytube), and encompasses the image as acquired, as well as the image dataafter being processed for display and/or storage or other use.

Tr refers to an image that is reconstructed from tomosynthesisprojection images Tp, for example, in the manner described in one ormore of U.S. Patent Application Publication No. 2010/0135558, and U.S.Pat. Nos. 7,760,924, 7,606,801, and 7,577,282, the disclosures of whichare fully incorporated by reference herein in their entirety, wherein aTr image represents a slice of the breast as it would appear in aprojection x ray image of that slice at any desired angle, not only atan angle used for acquiring Tp or Mp images.

Ms refers to synthesized 2D images, which simulate mammography images,such as a craniocaudal (CC) or mediolateral oblique (MLO) images, andare constructed using tomosynthesis projection images Tp, tomosynthesisreconstructed images Tr, or a combination thereof. Examples of methodsthat may be used to generate Ms images are described in theabove-incorporated U.S. Patent Application Publication No. 2010/0135558,and U.S. Pat. No. 7,760,924.

I_(MERGE) refers to a 2D image constructed by importing into a singleimage one or more objects and/or regions from any two or more of Mp, Ms,Tp or Tr images of a patient's breast, wherein an image from which anobject or region is imported into the merged image comprises a sourceimage for that object or region, and wherein objects or regions areimported into the merged image at X,Y coordinate locations correspondingto the X,Y coordinate locations of the objects or regions in theirrespective source image

The terms I_(MERGE), Tp, Tr, Ms and Mp each encompasses information, inwhatever form, that is sufficient to describe the respective image fordisplay, further processing, or storage. The respective I_(MERGE), Mp,Ms. Tp and Tr images are typically provided in digital form prior tobeing displayed, with each image being defined by information thatidentifies the properties of each pixel in a two-dimensional array ofpixels. The pixel values typically relate to respective measured,estimated, or computed responses to x rays of corresponding volumes inthe breast, i.e., voxels or columns of tissue. In a preferredembodiment, the geometry of the tomosynthesis images (Tr and Tp),mammography images (Ms and Mp) and the merged image I_(MERGE) arematched to a common coordinate system, as described in U.S. Pat. No.7,702,142, the disclosure of which is hereby incorporated by referencein its entirety. Unless otherwise specified, such coordinate systemmatching is assumed to be implemented with respect to the embodimentsdescribed in the ensuing detailed description of this patentspecification.

FIG. 1 illustrates the flow of data in an exemplary image generation anddisplay system, which incorporates the merged image generation anddisplay technology and features of the presently disclosed inventions.It should be understood that, while FIG. 1 illustrates a particularembodiment of a flow diagram with certain processes taking place in aparticular serial order or in parallel, various other embodiments of thepresently disclosed inventions are not limited to the performance of theimage processing steps in any particular order, unless so specified.

More particularly, the image generation and display system includes animage acquisition system 1 that acquires tomosynthesis image data forgenerating Tp images of a patient's breasts, using the respective threedimensional and/or tomosynthesis acquisition methods of any of thecurrently available systems. If the acquisition system is a combinedtomosynthesis/mammography system, Mp images may also be generated. Somededicated tomosynthesis systems or combined tomosynthesis/mammographysystems may be adapted to accept and store legacy mammogram images,(indicated by a dashed line and legend Mp_(legacy) in FIG. 1) in astorage device 2, which is preferably a DICOM-compliant PictureArchiving and Communication System (PACS) storage device. Followingacquisition, the tomosynthesis projection images Tp may also betransmitted to the storage device 2 (as shown in FIG. 1).

The Tp images are transmitted from either the acquisition system 1, orfrom the storage device 2, or both, to a computer system configured as areconstruction engine 3 that reconstructs the Tp images intoreconstructed image “slabs” Tr, representing breast slices of selectedthickness and at selected orientations, as disclosed in theabove-incorporated patents and application publication. The imaging anddisplay system 1 further includes a 2D synthesizer that operatessubstantially in parallel with the reconstruction engine for generating2D images that simulate mammograms taken at any orientation (e.g., CC orMLO) using a combination of one or more Tp and/or Tr images. Thesynthesized 2D images may be generated dynamically prior to display (asshown in FIG. 1) or may be stored in storage system 2 for later use. Thesynthesized 2D images are interchangeably referenced as T2d and Ms. Thereconstruction engine 3 and 2D synthesizer are preferably connected to adisplay system 5 via a fast transmission link. The originally acquiredMp and/or Tp images may also be forwarded to the display system 5 forconcurrent or toggled viewing with the respective Tr and/or Ms images bya medical professional.

Mode filters 7 a, 7 b are disposed between image acquisition and imagedisplay. Each of the filters 7 a and 7 b may additionally includecustomized filters for each type of image (i.e., Tp, Mp, Tr) arranged toidentify and highlight certain aspects of the respective image types. Inthis manner, each imaging mode can be tuned or configured in an optimalway for a specific purpose. The tuning or configuration may beautomatic, based on the type of the image, or may be defined by manualinput, for example through a user interface coupled to a display. In theillustrated embodiment of FIG. 1, the filters 7 a and 7 b are selectedto highlight particular characteristics of the images that are bestdisplayed in the respective imaging mode, for example, geared towardshighlighting masses or calcifications, or for making the merged image(described below) appear to be a particular image type, such as a 3Dreconstructed slice, or a 2D mammogram.

According to one aspect of the disclosed inventions, and as described ingreater detail herein, the system 1 includes an image merge processor 6that merges relevant image data obtained from a set of available sourceand synthesized images of a patient's breast to provide a merged 2Dimage I_(MERGE) for display. The set of available images used togenerate the merged image I_(MERGE) may include filtered and/orunfiltered Ms, Mp, Tr and/or Tp images. While FIG. 1 depicts all thesetypes of images being input into the image merge processor 6, it is alsoenvisioned within the scope of the disclosed inventions that the mergedimage may be manually configurable. For example, a user interface orpreset configuration may be provided and configured to allow a user toselect a particular group of two or more images or image types forgenerating a synthesized 2D image I_(MERGE) for display.

By way of illustration, a medical professional, such as a radiologist,may wish to merge two or more reconstructed tomosynthesis slices (orslabs) in order to provide a merged image showing the most readilydiscerned structures in the collective tomosynthesis image data in adisplayed synthesized 2D image, which essentially maps the tomosynthesisslices (or slabs) at a pixel wise granularity. Additionally oralternatively, the radiologist may combine a 2D mammogram image, whetherMp or Ms, with a 3D projection, or with selected reconstructed images,in order to obtain a customized merged image that highlights bothcalcifications and various tissue structures in the breast. Filtersapplied to each type of image can further highlight the types ofstructures or features in the merged image that are generally mostprevalent or most readily discerned in the respective source image type.Thus, one type of filter may be applied to mammography images tohighlight calcifications, while a different filter may be applied totomosynthesis slices to highlight masses, allowing both the highlightedcalcifications and highlighted tissue masses to be displayed in thesingle merged image. Filters may also provide the merged image with adesired look and feel; i.e., to make the merged image appear more like atomosynthesis or mammography image.

The display system 5 may be part of a standard acquisition workstation(e.g., of acquisition system 1), or of a standard (multi-display) reviewstation that is physically remote from the acquisition system 1. In someembodiments, a display connected via a communication network may beused, for example, a display of a personal computer or of a so-calledtablet, smart phone or other hand-held device. In any event, the display5 of the system is preferably able to display I_(MERGE), Ms, Mp and Tr(and/or Tp) images concurrently, e.g., in separate side-by-side monitorsof a review workstation, although the invention may still be implementedwith a single display monitor, by toggling between images.

To facilitate the detection/diagnosis process, Tr slices are preferablyreconstructed all to the same size for display, which can be the same asthe size of an Mp or Ms image of the breast, or they can be initiallyreconstructed to sizes determined by the fan shape of the x ray beamused in the acquisition, and then later converted to that same size byappropriate interpolation and/or extrapolation. In this manner, imagesof different types and from different sources can be displayed indesirable size and resolution. For example, an image can be displayed in(1) Fit To View Port mode, in which the size of the displayed image sizeis maximized such that the entire imaged breast tissue is visible, (2)True Size mode, in which a display pixel on the screen corresponds to apixel of the image, or (3) Right Size mode, in which the size of adisplayed image is adjusted so that it matches that of another imagebeing concurrently displayed, or with which the displayed image is, orcan be, toggled.

For example, if two images of the same breast are taken and are not thesame size, or do not have the same resolution, provisions may be made toautomatically or user-selectively increase or reduce the magnification(i.e., “zoom in” or “zoom out”) of one or both images, such that theyappear to be the same size when they are concurrently displayed, or as auser toggles between the images. Known interpolation, extrapolationand/or weighting techniques can be used to accomplish the re-sizingprocess, and known image processing technology can also be used to makeother characteristics of the displayed images similar in a way thatfacilitates detection/diagnosis. When viewing such resized images,according to one embodiment of the disclosed inventions, the mergedimage I_(MERGE) is automatically resized, accordingly.

Thus, the system 1, which is described as for purposes of illustrationand not limitation in this patent specification, is capable of receivingand displaying selectively tomosynthesis projection images Tp,tomosynthesis reconstruction images Tr, a synthesized mammogram imageMs, and/or mammogram (including contrast mammogram) images Mp, or anyone or sub combination of these image types. The system 1 employssoftware to convert (i.e., reconstruct) tomosynthesis images Tp intoimages Tr, software for synthesizing mammogram images Ms, and softwarefor merging a set of images to provide a merged image that displays, forevery region of the merged image, the most relevant feature in thatregion among all images in the source image set. For the purpose of thispatent specification, an object of interest or feature in a source imagemay be considered a ‘most relevant’ feature for inclusion in the mergedimage based upon the application of one or more CAD algorithms to thecollective source images, wherein the CAD algorithms assign numericalvalues, weights or thresholds, to pixels or regions of the respectivesource images based upon identified/detected objects and features ofinterest within the respective region or between features or, ininstances when the merged image is generated directly from thesynthesized image without CAD assistance, simply the pixel value, weightor other threshold associated with a pixel or region of the image. Theobjects and features of interest may include, for example, spiculatedlesions, calcifications, and the like. Various systems and methods arecurrently well known for computerized detection of abnormalities inradiographic images, such as those disclosed by Giger et al. inRadioGraphics, May 1993, pp. 647-656; Giger et al. in Proceedings ofSPIE, Vol. 1445 (1991), pp. 101-103; U.S. Pat. Nos. 4,907,156,5,133,020, 5,343,390, and 5,491,627, each of which being herebyincorporated by reference in its entirety.

FIG. 2 is a diagram which pictorially illustrates the merging of imagedata from a tomosynthesis reconstruction image data set Tr, comprisingtomosynthesis slices 10A to 10N, with image data from a mammogram 20, inthis case a synthesized mammogram Ms. For ease of description, filtersare not shown in this example. The tomosynthesis image data set Tr andsynthesized mammogram Ms are forwarded to the region compare and imagemerge processor 6, which evaluates each of the source images for which amerged image is to be generated (i.e., whether automatically, or basedon a specific user commend) in order to (1) identify the objects andfeatures of interest in each image for those that may be considered a‘most relevant’ feature for possible inclusion in the merged image basedupon the application of one or more CAD algorithms (as described above),(2) identifies respective pixel regions in the images that contain theidentified features, and (3) thereafter compares the images on a regionby region basis, searching for that image with the most desirabledisplay data for each respective region.

As discussed above, the image with the most desirable display data maybe an image with a highest pixel value, a lowest pixel value, or whichhas been assigned a threshold value or weight based on the applicationof a CAD algorithm to the image. When the image with the most desirabledisplay data for that region is identified, the pixels of that regionare copied over to the corresponding region of the merged image. Forexample, as shown in FIG. 2, region 36M from image Ms is written toregion 361. Region 35 of tomosynthesis slice 10A is copied to region 351of the merged image. Although the regions of FIG. 2 are shown aspre-defined grid regions, it is not necessary that regions bepre-defined in this manner. Rather, according to one aspect of thedisclosed inventions, the boundaries of the regions may be dynamicallyidentified during the region compare and image generation process byperforming comparisons at pixel or multi-pixel granularities. By way ofillustration, FIG. 3 illustrates a merged image 50, which has beenconstructed via the combinations of numerous regions of different sourceimages, at arbitrary region boundaries, for example, which may beidentified according to the detection of particular features within therespective source images.

FIG. 4 is a flow diagram provided to illustrate exemplary steps that maybe performed in an image merge process carried out in accordance withone embodiment of the disclosed inventions. At step 62, an image dataset is acquired. The image data set may be acquired by a tomosynthesisacquisition system, a combination tomosynthesis/mammography system, orby retrieving pre-existing image data from a storage device, whetherlocally or remotely located relative to an image display device. At step64, a user may optionally select a merge mode, wherein the user maydesignate (1) which images are to be used for the source image set togenerate the merged image, (2) whether to highlight certain features inthe merged image, such as calcifications, spiculated lesions or masses,and (3) whether to display the image as a lower resolution tomosynthesisimage, etc. At step 66, the images that are to be merged to generate themerged image are mapped to a common coordinate system, for example, asdescribed in the above-incorporated U.S. Pat. No. 7,702,142. Othermethods of matching images of different coordinate systems mayalternatively be used. At step 72, the process of comparing regionsamong the different images begins. At step 74, each I_(MERGE) region ispopulated with the pixels of the region of an image from the sourceimage set having the most desirable pixels, value, or pattern. Theprocess of populating regions continues until it is determined, at step76, that all regions have been evaluated, at which point the mergedimage is ready for display.

Once the merged image is generated, it may be used to assist in thenavigation through a tomosynthesis image data stack from which the mergeimage was generated. Such navigation is a two-step process comprisingselection of various objects of interest, and display of correspondingtomosynthesis images that are the source of such objects of interest inthe merged image. By way of example, FIG. 5A and FIG. 5B illustrate twoviews of a display 80. The first view of display 80 shown in FIG. 5Aillustrates a merged image 82, having regions sourced by different onesof an acquired or synthesized image set. FIG. 5B illustrates aparticular feature enabled by the presently disclosed inventions,whereby a user may select a region or area 83 within the merged image82, and the resulting image source 84 for that area is presented to theuser.

The presently disclosed inventions envision many different mechanismsfor selection of the objects of interest and corresponding display ofthe respective source images corresponding; although it is to beunderstood that the disclosed inventions are not limited to thosedescribed herein. For example, the selection of a region or area withinthe merged image may include a selection of a CAD mark, or alternativelya selection of a particular feature of interest to the reviewer.Although in both instances the most relevant slices are made availableto the user, the mechanics behind the processes differ. One suchpreferred mechanism is illustrated in FIG. 2. As the regions of themerged image are populated, a merge (or “guidance”) map 40 is alsoconstructed. The merge map stores, for each region of the merged image,an identifier of the image from which the region is sourced. Therefore,as shown in FIG. 2, the Ms identifier is stored in region 36, while the10A TR slice identifier is stored in region 35. As will be described inmore detail herein, the merged map may be used during the display of themerged image to permit fast viewing of the respective source image(s)for user-selected regions or objects of interest.

Selection Using CAD Marks:

In addition or alternatively to use of a merge/guidance map, if themerged image is presented with a CAD overlay, the CAD overlay mayinclude either CAD marks derived from 3D data, or CAD marks derived from2D data (if the system has the ability to obtain 2D data). CAD marksderived from 3D data generally include, as part of the data objectassociated with the mark, identifiers of one or more slices whichcontributed to the generation of the 3D mark. When the merged image isoverlaid with 3D CAD data, selection of the CAD mark results in theretrieval of the series of slices that contributed to the mark. In oneembodiment, the central image slice is displayed; in alternateembodiments, the image slice having the highest weight is displayed; andin a still further alternate embodiment, the image slice having theleast visual noise (i.e., the clearest image) is displayed.

Selection by Objects of Interest:

As an alternate to selecting by CAD marks, a mechanism is provided forallowing a user to select any object on a merged image, whether it is aCAD mark, or a feature of interest, such as any abnormality orirregularity in the image. In one embodiment, the user or system mayselect a region, using for example a mouse click for a single pixelarea, or a click and drag action to select a larger region.Alternatively, the user may be provided with a selection of graphicalframes of various or variable sizes, and have the ability to move theframe to different locations within the merged image to select areaswhen it is desired to view additional tomosynthesis image slices. Inresponse to such a selection, the particular image slice for initialdisplay may be selected in a variety of ways.

For example, an image slice could be selected based on the weighting ofits associated pixel within the selected region. Or a particular imageslice may be selected because a particular feature which is selected, orwhich is near a pixel or region that is selected, is best viewed in theselected image slice, e.g., provides the clearest view of that region.Thus, the identification of a particular image slice that is mostrelevant to a selected pixel or region may utilize pixel informationthat surrounds the selected object, for example, using region growingtechniques known to those in the art. Thus, pixels that neighbor theselected pixel or region are included in the evaluation for relevantslices if the pixels have a characteristic that satisfies a certainthreshold established by the user; for example, including but notlimited to the pixels having a particular weight, or being arranged in aparticular pattern, etc.

Alternatively, a group of image slices may be selected, e.g., asuccessive order of image slices, with a central slice or most heavilyweighted slice being first presented. As described above, alternativelythe image slice within the group having the least noise, i.e., theclearest slice, may be provided. In addition, the selection of an imageslice for presentation may also take into account a desiredvisualization mode. Thus, if the user-specified purpose is to visualizecalcifications, an image slice having calcification features may bepresented ahead of another slice within the group having a lessercalcification characteristic.

It will be appreciated that the disclosed and described systems andmethods in this patent specification are designed to condense the imageinformation made available from a tomosynthesis reconstruction volume(or “stack”) containing a patient's 3D breast image data down to asingle, synthesized 2D image, similar to a conventional 2D mammographicimage. By reviewing this synthesized 2D image concurrently with the 3Dtomosynthesis stack, it is possible to provide a much more efficient andaccurate review of the patient's breast tissue. This is because thesynthesized 2D merged image can act as a guidance-map, so that themedical professional reviewing the images can focus on the synthesized2D image for detecting any objects or regions of interest that meritfurther review, and the system can provide immediate, automatednavigation to a “best” corresponding tomosynthesis image slice (or asubset of adjacent tomosynthesis slices) to allow the medicalprofessional to conduct this further review, verify and evaluate thefinding. Thus, it is preferred, although not required for practicing allembodiments of the disclosed inventions, for the medical professional toemploy a user interface that can display a respective synthesized 2Dmerged image along-side the tomosynthesis volume image slices, forconcurrent viewing of both.

FIG. 6 illustrates one exemplary process 180 for retrieving andpresenting a Tr image slice in response to user-selection of an objectof interest in a merged image, which may be implemented using a softwareprogram according to one embodiment of the presently disclosedinventions. The process 180 operates, in response to a selection of anobject of interest in a merged image at step 182. At step 184, theprocess determines whether the selected object is a CAD mark or anon-CAD mark feature of interest. If it is a CAD mark, at step 185, theTr slices related to the CAD mark are retrieved. At step 189, one of theTr slices is selected and presented for display based on at least one ofits relative position in the stack, relative weight of the voxel valueof the slice, a selected visualization mode, etc. If, at step 184, theprocess determines that the selected object was a non-CAD mark featureof interest, then at step 186, the source Tr images associated with theselected region are evaluated, and a particular Tr source is selectedfor display based on its relative voxel value as compared to voxelvalues in other Tr sources that map to the region. It is noted that theTr sources that contribute to pixel values within a selected region maybe intermittently spaced within the 3D tomosynthesis volume. Thus, whenthe most relevant Tr source image is selected, it may be presentedeither alone, or as part of a stack of images together with one or moreneighboring Tr slice images. The most relevant Tr source may be thepresented image, or alternatively another image in the stack associatedwith the most relevant image may be first presented, for example if thatparticular image is clearer.

FIG. 7 depicts another process that may be software-implemented forusing a synthesized 2D image for navigating a 3D tomosynthesis imagestack (“tomosynthesis stack”), according to another embodiment of thepresently disclosed inventions. At initiation or activation 90, theprocess includes, at step 92, constructing a tomosynthesis image sliceindex map, wherein the pixel locations of the synthesized 2D image aremapped to corresponding pixel locations in pertinent image slices of thetomosynthesis stack. In particular, the tomosynthesis stack index mapincludes identifying information of selected tomosynthesis slice imagesfrom the breast volume stack that are source images or that otherwisecontain a most similar representation of regions and/or objectsdisplayed in the synthesized 2D image. The tomosynthesis stack index mapis preferably generated prior to when a medical professional is ready toconduct his or her review of the breast image data. The details forconstructing the tomosynthesis stack index map, in accordance with onepreferred embodiment, are described below in conjunction with FIG. 8.

The synthesized 2D image is displayed to the medical professional(interchangeably referred to as the “user” of the described system),typically on a workstation having side-by-side monitors as depicted inFIGS. 9-11. Depending on how the user has configured the workstation,when initiating review of particular patient breast image data, only thesynthesized 2D image may be presented, e.g., on the left-hand-sidemonitor, with the right-hand-side monitor being blank, or perhapsdepicting a first or middle image slice from the tomosynthesis stack,preferably depending on a user-selectable configuration. In oneembodiment, the system will initially display the synthesized 2D imageon the left-hand-side monitor, and a “most relevant” one of thetomosynthesis slice images on the right-hand-side monitor, which wasdetermined by the system based upon the displayed tomosynthesis slicebeing most similar in appearance to the synthesized 2D image, or havingthe relatively most interesting objects, out of the tomosynthesis imagestack for the entire breast volume.

Thereafter, the medical professional (user) may use the user-interfaceto activate the navigational capability of the system. In particular, atstep 94, the user may affirmatively input a command to select aparticular object or region in the displayed synthesized 2D image.Alternatively, the system may be configured so that the user merelypositions a “pointer,” e.g., a movable cross or arrowhead that iscontrolled using a mouse or similar input device), overlying an objector region in the displayed synthesized 2D image, thereby “indicating” aninterest in the item. In response to the received command or indication,using the index map, the system may easily retrieve, at step 96, anddisplay on the right-hand-side monitor, at step 98, the tomosynthesisslice that is either the direct source of the user selected/indicatedobject or region, or which otherwise contains a most similarrepresentation of the object or region as depicted in the displayed 2Dimage. Additionally and/or alternatively, the system may be configuredfor concurrently displaying a respective source image and/or mostsimilar representation of a tissue structure or region that correspondsto a given location of a user movable input device in the displayedsynthesized 2D image.

The plurality of 2D and/or 3D images from which a synthesized 2D imageis generated may include tomosynthesis projection images, tomosynthesisreconstruction slices, mammography images, contrast enhanced mammographyimages, synthesized 2D images, and combinations thereof. It will beappreciated that the synthesized 2D image advantageously incorporatesthe most relevant information from each of the underlying acquired andcomputer generated image data sets of the patient's breast. Thus,different regions of pixels in the displayed synthesized 2D image may besourced from corresponding different images in the underlying image dataset, depending on which underlying image is best for viewing an objectof interest, e.g., a mass or a calcification, in the respective region.The particular regions may be identified statically, i.e., within aparticular grid, or dynamically, i.e., based on identified objects ofinterest, and may range in granularity from as little as one pixel, toall pixels in the respective image. In one embodiment, priority is givento first importing into a merged image under construction those regionscontaining one or more specific tissue structures of interest in theimages of a tomosynthesis image data set (or “stack”), and thereafterpopulating the remaining regions of the merged image with the otherwisemost relevant regions from the images, as described above.

The user interface may additionally include features to enable themedical professional to manipulate the presented tomosynthesis data, forexample, to allow the medical professional to scan through adjacentimage slices of the tomosynthesis stack, or to further zoom (magnify)into a selected region, to place markers, or alternatively to applyfilters or other image processing techniques to the image data. In thismanner, the medical professional may quickly review a large stack oftomosynthesis data by utilizing a synthesized 2D image for navigationpurposes, thereby increasing the performance and efficiency of breastcancer screening and diagnosis. According to a further aspect of thedisclosed inventions, it has been determined or otherwise appreciatedthat particular types of images may include or be superior for viewingdifferent types of relevant information. For example, calcifications aretypically best visualized in 2D mammograms, while masses are typicallybest visualized using 3D reconstructed images.

Thus, in one embodiment of the disclosed inventions, different filtersare applied to each of the different types of underlying 2D and/or 3Dimages in the image data set used to generate the merged image, thefilters selected to highlight particular characteristics of the imagesthat are best displayed in the respective imaging mode. Appropriatefiltering of the images prior to generating the merged image helpsensure that the final merged image includes the most relevantinformation that can be obtained from all the underlying image types.Additionally and/or alternatively, the type of filtering performed forthe various images may be defined via user input, which permits a userto select a ‘merge mode’, for example, geared towards highlightingmasses, calcifications, or for making the merged image appear to be aparticular image type, such as a 3D reconstructed slice, or a 2Dmammogram.

Synthesizing the 2D image may be accomplished in a variety of ways. Forexample, in one embodiment, general purpose image filtering algorithmsare used to identify features within each of the respective 2D and 3Dimages, and a user may select whether to use 2D filtered data or 3Dfiltered data to generate the merged image. Alternatively, 2D or 3Dfiltered data may be automatically selected in accordance with aparticular visualization mode that has been user selected; for example,2D filtered data may be automatically selected by the system forcalcification visualization mode, while 3D filtered data may beautomatically selected by the system for mass visualization modes. Inone embodiment, two different merged images may be constructed, one foreach mode; alternatively, a single merged image may be constructed thattakes into account the respective filtered image data results from allavailable image types.

In one embodiment, features (representing potential objects of interest)are identified in the available source images and thereafter weighted,e.g., on a pixel by pixel or region by region basis in each respectiveimage. A 2D image is then constructed by incorporating the respectiveregions having the most significant weight in individual images of theavailable source images. The size of the region may vary in granularityfrom one pixel to many (or even all) pixels of the respective image, andmay be statically pre-defined, or may have margins that vary inaccordance with the varying thresholds of the source images. Thesynthesized (aka “merged”) image may be pre-processed and stored as aDICOM object following tomosynthesis acquisition, and thereafterforwarded with the reconstruction data for subsequent review by amedical professional. Such an arrangement removes the need to forwardweighting information for each reconstructed slice. Alternatively, thestored DICOM object may include the weighting information, allowing themerged image to be dynamically constructed in response to a request fora synthesized 2D image at the medical professional's work station. Inone embodiment, both the weighting information and the synthesized 2Dimage may be provided in the DICOM object, allowing presentation of adefault merged image, while still enabling customization according tothe personal workflow of the reviewer. To be clear, the weightinginformation can be stored with the image itself, and need not be aseparate file.

It is realized that the visualization of the synthesized 2D images mayhave some drawbacks. For example, there may be neighboring regions inthe merged image which exhibit bright calcifications, but which in factare sourced from image slices that are distant from one another in the zplane. Therefore, what may appear to be a cluster ofmicro-calcifications in the 2D image may, in fact, be individualcalcifications that are distributed (i.e., along the z-axis) throughoutthe breast and thus do not actually represent a micro-calcificationcluster that requires further review. Thus, according to a furtheraspect of the disclosed inventions, a ‘cluster spread indicator’ may beprovided with the synthesized 2D image, which visually indicates thedistribution of calcifications along the z-plane, allowing the medicalprofessional to quickly assess whether a group of calcificationscomprise a calcification cluster.

In some instances, the system may determine based on the index mapinformation that more than one tomosynthesis image slice should bedisplayed for a selected/indicated object type or region, for example, aspiculated mass. In such instances, a series of two or more adjacenttomosynthesis slices are displayed, one after the other, at a timinginterval that is preferably user selected. As will be additionallydescribed herein, the user may select or indicate more than one objector region in a given synthesized 2D image. Once the user has completedhis or her review of the displayed tomosynthesis slice(s), the processis complete (at step 100) for the particular breast image data.

As previously pointed out, while various image processing techniques maybe employed for providing the this navigational functionality, in apreferred embodiment, the system is preferably configured for, and themethod further includes, generating an index map comprising identifyinginformation of selected images of the plurality of 2D and/or 3D imagesthat are source images or that otherwise contain a most similarrepresentation of regions and/or objects displayed in the synthesized 2Dimage. The index map can thereafter be used by the system for to greatlyreduce the time needed to navigate through the images, e.g., atomosynthesis volume stack of the breast image volume.

An implementation of one preferred process 102 for generating an indexmap will now be described in conjunction with the flow diagram shown inFIG. 8. Two parallel processes are initially employed. In one process,the image data contained in the synthesized 2D image 104 is mapped toselected tomosynthesis image slices of a 3D volume 106 to construct a“generic” index map 108. In particular, the pixel locations in the 2Dimage 104 is mapped to the pixel locations in the respective 3D(tomosynthesis) images 106 based entirely on image similarity, akin tothe pieces of a jigsaw puzzle In other words, the generic index map 108is based entirely on best-fit matching of the appearance of the data inthe 2D image to the appearance of the data in the respective 3D images,wherein the slice identification and X,Y coordinates of a 3D imagehaving a most similarly appearing pixel region to a corresponding X,Yregion in the 2D region is selected. Potential importance of therespective objects and features in the synthesized 2D image is not takeninto account for constructing the generic index map 108.

However, in parallel with the creation of the generic index map 108, anobject type index map 114 is generated, in which individual objecttypes, designated as 110-1 to 110-n in FIG. 8, in the synthesized 2Dimage are prioritized and assigned weighted values to influence theselection of the best corresponding 3D tomosynthesis image slice. Inparticular, an individual object type index map, designated as 112-1 to112-n in FIG. 8, is generated for each object type identified in thesynthesized 2D image, e.g., blob density, spiculated masses,micro-calcifications, etc. The individual object type index maps 112-1to 112-n are then combined to construct the full object type index map114, which is then blended, at step 116, with the generic index map 108to provide a composite index map 120, wherein the object type image datais prioritized relative to the generic image data. The composite indexmap 120 is then used by the system for navigating the image slices ofthe 3D volume 106 in response to a selected or indicated location on the2D image 104. In this manner, different object types having overlappingX,Y coordinates, i.e., due to their location at different z-axispositions in the volumetric breast image, can nevertheless be separatelynavigated for selective viewing, since separate mapping indexes areprovided (See below example with respect to FIGS. 10 and 11).

As noted above, in various embodiments, an object or region may beautomatically highlighted in the synthesized 2D image and/or displayedat least portion of the one or more images from the plurality.Additionally and/or alternatively, an object or region in thesynthesized 2D image and/or displayed at least portion of the one ormore images from the plurality may be highlighted in response to afurther received user command or to certain user activity detectedthrough the user interface. By way of non-limiting example, an object orregion may is highlighted by a contour line representing a boundary ofthe highlighted object or region. Preferably, the object or region ishighlighted in a manner indicating that the highlighted object or regionis or contains a specified type of tissue structure.

By way of illustration, FIG. 9 depicts an exemplary work station display122, including a left-hand side monitor 124 (“C-View”) displaying asynthesized 2D image 132 of a patient's breast. The synthesized 2D image132 includes a highlighted tissue structure 134, wherein thehighlighting is in the form of a contour line that represents a boundaryof the tissue structure. As noted above, this highlighting may have beendone automatically by the system, e.g., at the time the 2D image 132 isinitially displayed, or only in response to a specific user command orindication, e.g., by hovering a pointer over the object 134 in the 2Dimage 132. The work station display 122 also includes a right-hand sidemonitor 126 displaying the respective tomosynthesis image 136 (which isslice no. 18 of the tomosynthesis volume stack, as indicated in thelower right hand side of the monitor 126), which is the source image orwhich otherwise provides a most similar view of the highlighted tissuestructure 134 as seen in the synthesized image 132. In particular, theuser interface associated with the display 122 allows for a user toselect or otherwise indicate a location on the synthesized 2D image 132,e.g., by displaying a pointer, a cross, a circle, or other similargeometrical object, and then input a certain command type (e.g., mouseclick) that will be recognized by the system as a request from the userto have the corresponding source or otherwise most similar tomosynthesisslice(s) depicting the region or object underlying the pointer displayedin monitor 126.

FIG. 10 depicts the work station display 122, wherein a differentsynthesized 2D breast image 142 is displayed in the left-hand sideC-View monitor 124. The synthesized 2D image 142 includes a highlightedtissue structure 144, wherein the highlighting is in the form of ageometric shape, in this case a circle, to indicate that the object 144is a spiculated mass. Again, this highlighting may have been doneautomatically by the system, e.g., at the time the 2D image 142 isinitially displayed, or only in response to a specific user command orindication, e.g., by hovering a pointer over the object 144 in the 2Dimage 142. The right-hand side monitor 126 is displaying the respectivetomosynthesis image 146 (which is slice no. 33 of the tomosynthesisvolume stack, as indicated in the lower right hand side of the monitor126), which is the source image or which otherwise provides a mostsimilar view of the highlighted tissue structure 144 as seen in thesynthesized image 132.

It should be appreciated that there will be instances in which themapping between an object or region in the merged 2D image to therespective object or region in the displayed (i.e., source or “best”)image may not necessarily be 1-to-1, and will possibly be “1-to-many” incertain circumstances, for example, when multiple line structures ondifferent tomosynthesis image slices combine together to form aline-crossing structures in the synthesized 2D image. By way of example,FIG. 11 depicts the user work station display 122, including the samesynthesized 2D breast image 142 as displayed in FIG. 10, but nowhighlighting a region 154 containing micro-calcifications, with theright-hand side monitor displaying the tomosynthesis image slice 156(which is slice no. 29 of the tomosynthesis volume stack, as indicatedin the lower right hand side of the monitor 126), from which thehighlighted region 154 was imported into the 2D image 142, or whichotherwise provides a best view of the micro-calcifications. Inparticular, because the spiculated mass structure 144 and region ofmicro-calcifications 154 are in very close proximity in FIG. 142, adifferent one may be highlighted depending on a specific user command(e.g., to highlight a certain tissue type), or by slight adjustment ofthe position of the pointer of the user interface.

As explained above, this above described examples with respect to FIGS.9-11 are readily accomplished by the index map constructed at the sametime (or after—depending on the system implementation) the synthesized2D image is generated. Alternatively, if no index map is available, forany given such user selected/specified point/location on the 2D imagedisplayed in the left-hand-side monitor 124, the system may execute analgorithm to automatically compute the best corresponding image (i.e.,X,Y and Z) within the tomosynthesis stack for display on theright-hand-side monitor 126. A “tomosynthesis slice indicator” mayoptionally be provided on the left-hand-side monitor 124, whichindicates which tomosynthesis slice number (numbers) would be displayedon the right-hand-side monitor 126 based on a current location of a usercurser on the 2D image. With this feature, the reviewer need not bedistracted by constantly changing image displays on the right-hand-sidemonitor 126, while still providing the reviewer with an understanding ofthe z-axis location in the tomosynthesis volume stack of a particularobject in the 2D image.

In accordance with a further aspect of the disclosed inventions, theavailable features of the user interface may be extended to function,not only based point/location of the merged image, but also based in asimilar fashion on a structure/object/region. For example, particularobjects or region(s) in the merged image may be automaticallyhighlighted when displayed, based on the system recognition of possibleinterest in the respective objects, or of objects located in therespective region(s). In one embodiment, shown in FIG. 8, thishighlighting is in the form of a contour line 108 that represents aboundary of a highlighted tissue structure. A contour line may besimilarly used to highlight regions of interest in the displayed image,e.g., containing a number of calcification structures. In someembodiments, the system is configured to allow the user to “draw” acontour line on the merged image as a way of selecting or otherwiseindicating an object or region of interest for causing the system toconcurrently display one or more underlying source images of theselected or indicated object or region.

In preferred embodiments, the system employs known image processingtechniques to identify different breast tissue structures in the varioussource images, and highlight them in the merged image, in particular,tissue structures comprising or related to abnormal objects, such asmicro-calcification clusters, round-or-lobulated masses, spiculatedmasses, architectural distortions, etc.; as well as benign tissuestructures comprising or related to normal breast tissues, such aslinear tissues, cysts, lymph nodes, blood vessels, etc. Furthermore, anobject or region consisting of or containing a first type of tissuestructure may be highlighted in a first manner in the displayed mergedimage, and an object or region consisting or containing a second type oftissue structure may be highlighted in a second manner different fromthe first manner in the displayed merged image.

In various embodiments, the user may input a command through the userinterface selecting or otherwise identifying a certain type of tissuestructure, and, in response to the received command, the system performsone or both of (i) automatically highlighting in the displayed mergedimage objects comprising the selected type of tissue structure and/orregions containing one or more objects comprising the selected type oftissue structure, and (ii) automatically concurrently displaying therespective source slice (or otherwise the slice with best depiction of)a tissue structure of the selected type in the breast image data, e.g.,a most prominent one of the selected tissue structure type based on acomparison, if more than one is detected in the source image stack.Thus, when the user “click” on (or very close to) a micro-calcificationspot/cluster in the merged 2D image, and the system automaticallyconcurrently displays the source (or otherwise best) tomosynthesis imageslice including the corresponding micro-calcification in 3D. By way ofanother example, a user can select (through the user interface) a regionin the 2D merged image that has the appearance with radiating linepatterns (often an indication of spiculated masses), and the system willconcurrently display the source (or otherwise best) 3D tomosynthesisslice, or perhaps to a series of consecutive tomosynthesis slices, forviewing the radiating line patterns.

In various embodiments, the user may input a command through the userinterface, activating dynamic display functionality, wherein the systemautomatically highlights those objects and tissue structures that(dynamically) correspond to the location of a user movable input devicein the displayed merged image. In such embodiments, the system mayfurther comprise automatically concurrently displaying a respectivesource image of a highlighted selected tissue structure that correspondsto a given location of a user movable input device in the displayedmerged image, again, on a dynamic basis.

In one embodiment, the system can be activated to provide a “shadow”cursor is displayed on the right-hand-side monitor 126, in a locationcorresponding to the same (x,y) location as the user's actual curser onthe left-hand-side monitor 124, so that moving the curser around in the2D image moves the shadow curser in the tomosynthesis image at same X,Ycoordinates. The reverse can also be implemented, i.e., with the activeuser curser operable in the right-hand monitor 126, and the show curserin the left-hand monitor 124. In one implementation, this dynamicdisplay feature allows the system to follow the user's point ofinterest, e.g. mouse cursor location, in the 2d merged image, anddynamically display/highlight the most “meaningful” region(s) underneathin real time. For example, the user can move the mouse (without clickingany button) over a blood vessel, and the system will instantly highlightthe vessel contour.

It should be appreciated that the presently disclosed inventions may beextended such that, rather than generate merely a synthesized 2D imageand associated index/guidance map, the mapping concepts described hereinmay be extended to generate a fully mapped 3D volume, with each of thevoxels in the mapped volume storing information related to theassociated tomosynthesis slices(s) sourcing the particular voxel. Forexample, in one embodiment, the volume may be projected onto a fixedcoordinate system, regardless of the actual volume of the breast.Projecting the volume to a fixed coordinate system in this mannerfacilitates processing of the image data, in particular, simplifying thecorrelation of voxels obtained during different acquisitions. Forexample, facilitating correlation of voxels in a 3D volume obtained froma CC acquisition of a breast with voxels in a volume obtained from anMLO acquisition of the same breast. In such an arrangement, one or more3D maps may be provided, for example, to map from voxels in one slice ofa 3D volume acquired via CC to one or more corresponding voxels inanother volume, for example acquired via an MLO view. Such anarrangement facilitates comparison of slices obtained from differentacquisitions that relate to a similar feature of interest within thebreast volume, essentially permitting the medical professional to obtaina multi-planar review of a region of interest.

Having described exemplary embodiments, it can be appreciated that theexamples described above and depicted in the accompanying figures areonly illustrative, and that other embodiments and examples also areencompassed within the scope of the appended claims. For example, whilethe flow diagrams provided in the accompanying figures are illustrativeof exemplary steps; the overall image merge process may be achieved in avariety of manners using other data merge methods known in the art. Thesystem block diagrams are similarly representative only, illustratingfunctional delineations that are not to be viewed as limitingrequirements of the disclosed inventions. Thus the above specificembodiments are illustrative, and many variations can be introduced onthese embodiments without departing from the scope of the appendedclaims.

1. (canceled)
 2. A method for processing breast image data, the methodcomprising: obtaining a plurality of images of a breast; applying a CADalgorithm to each image of the plurality of images, wherein applying theCAD algorithm to each image comprises identifying a feature in aplurality of feature-containing images of the plurality of images;generating a synthesized 2D mammography image of the breast, wherein thesynthesized 2D mammography image comprises the feature in the pluralityof feature-containing images; displaying the synthesized 2D mammographyimage and a CAD overlay on the 2D synthesized image, wherein the CADoverlay comprises a CAD mark; receiving a selection of the CAD mark; andbased on the selection, retrieving at least one feature-containing imageof the plurality of feature-containing images associated with theselected CAD mark.
 3. The method of claim 2, wherein the CAD mark isobtained from at least one of a tomosynthesis image data and a 2D imagedata of the plurality of images of the breast.
 4. The method of claim 2,wherein the selected CAD mark is associated with at least one featuredisplayed in the synthesized 2D mammography image.
 5. The method ofclaim 4, wherein the retrieved at least one feature-containing image ofthe plurality of feature-containing images comprises a plurality offeature-containing images, wherein each of the plurality of retrievedfeature-containing images contains the at least one feature.
 6. Themethod of claim 5, further comprising displaying the retrieved at leastone feature-containing image.
 7. The method of claim 5, furthercomprising assigning at least one of a numerical value, a weight, and athreshold to each of the plurality of feature-containing images.
 8. Themethod of claim 7, further comprising displaying a single image of theplurality of feature-containing images containing the at least onefeature, wherein the displayed single image comprises a singlefeature-containing image of the plurality of feature-containing imageshaving a predetermined weight.
 9. The method of claim 8, wherein thepredetermined weight is a highest weight.
 10. The method of claim 7,further comprising displaying a single image of the plurality offeature-containing images containing the at least one feature, whereinthe displayed single image comprises a single feature-containing imageof the plurality of feature-containing images having a lowest visualnoise.
 11. A method for processing breast image data, the methodcomprising: obtaining a plurality of images of a breast; applying a CADalgorithm to each image of the plurality of images, wherein applying theCAD algorithm to each image comprises identifying a plurality offeatures in a plurality of feature-containing images of the plurality ofimages; applying an assignment to each of the identified plurality offeatures; generating a synthesized 2D mammography image of the breast,wherein the synthesized 2D mammography image comprises the identifiedplurality of features; displaying the synthesized 2D mammography image;highlighting with a first highlight a first feature of the identifiedplurality of features based on the assignment of the first feature; andhighlighting with a second highlight a second feature of the identifiedplurality of features based on the assignment of the second feature. 12.The method of claim 11, wherein the first highlight is different thanthe second highlight.
 13. The method of claim 11, wherein the assignmentcomprises at least one of a numerical value, a weight, and a threshold.14. The method of claim 11, wherein the at least one of a numericalvalue, a weight, and a threshold comprises a maximum.
 15. The method ofclaim 11, wherein at least one of the first highlight and the secondhighlight comprises a contour line.
 16. The method of claim 11, whereinthe first highlight comprises a boundary of the first feature.
 17. Themethod of claim 11, wherein at least one of the first highlight and thesecond highlight comprises a geometric shape.
 18. The method of claim11, wherein at least one of the first highlight and the second highlightis associated with a particular tissue structure.
 19. The method ofclaim 11, further comprising receiving a selection of the firsthighlight.
 20. The method of claim 19, further comprising displaying atleast one of the plurality of feature-containing images based on theselection.